Skip Headers

Oracle® Universal Installer and OPatch User's Guide 10g Release 2 (10.2) for Windows and UNIX Part Number B16227-02 |
Home | Book List | Contents | Index | Master Index | Feedback

Previous | Next
| View PDF

# 6 Cluster Environment Installations

A cluster installation uses Oracle Universal Installer to install software on the nodes of a cluster that are network reachable and bound together by Oracle Clusterware. You can use Oracle Universal Installer to extend the Oracle home of a product installation to include additional nodes on the cluster.

This chapter includes the following sections:

- Oracle Universal Installer and Real Application Clusters

- General System Installation Requirements for Real Application Clusters

- Cluster Setup and Pre-Installation Configuration Tasks for Real Application Clusters

- Configuring Storage for Oracle Clusterware

- Installing Oracle Clusterware on UNIX

- Installing Oracle Clusterware on Windows

- Adding Additional Nodes to the Cluster for Mass Deployment

- Installing Product Software on a Cluster

- Patchsets and Upgrades

- Post Installation Tasks

- De-installing Real Application Clusters Software

- Troubleshooting Real Application Clusters / Oracle Clusterware Installation

## Oracle Universal Installer and Real Application Clusters

The Oracle Universal Installer facilitates the installation of Oracle Clusterware. In most cases, you use the graphical user interface (GUI) provided by the Oracle Universal Installer to install the software. However, you can also use the Oracle Universal Installer to complete non-interactive (or "silent") installations, without using the GUI.

The Oracle inventory maintains records of Oracle software versions and patches. Each installation has a Central Inventory where the Oracle home is registered. Oracle software installations have a local home directory, whose path location is recorded in the Central Inventory. The local inventory directory for each Oracle software installation contains a list of components and applied interim patches associated with that software. Because your Oracle software installation can be corrupted by faulty inventory information, the Oracle Universal Installer must perform all read and write operations on Oracle inventories. Ensure that you do not modify files in the Central Inventory or the Oracle home inventory.

When you install Oracle Clusterware or Real Application Clusters, the Oracle Universal Installer copies the Oracle software onto the node from which you are running it. If your Oracle home is not on a shared file system, then the Oracle Universal Installer propagates the software onto the other nodes that you have selected to be part of your Oracle Universal Installer installation session. The Oracle inventory maintains a list of each node that is a member of the Real Application Clusters database, and lists the paths to each node's Oracle home. This is used to maintain patches and updates for each member node of the Real Application Clusters database.

When the Oracle Universal Installer installs the Oracle software, Oracle recommends that you select a preconfigured database, or use the Database Configuration Assistant (DBCA) interactively to create your cluster database. You can also manually create your database as described in procedures posted on the Oracle Technical Network, which is at the following URL:

    http://www.oracle.com/technology/index.htm

Oracle recommends that you use Automatic Storage Management (ASM). If you are not using ASM, or if you are not using a cluster file system, then configure shared raw devices before you create your database.

Oracle software provides additional components to operate Real Application Clusters. Some of the Real Application Clusters-specific components include:

- Oracle Clusterware

- Installed Real Application Clusters Components

### Oracle Clusterware

The Oracle Universal Installer installs Oracle Clusterware on each node on which the Oracle Universal Installer detects that third-party vendor clusterware is present. If third-party vendor clusterware is not present, then you must use the Oracle Universal Installer to enter the nodes on which you want the Oracle Clusterware to be installed. The Oracle Clusterware home can be either shared by all nodes, or private to each node, depending on your responses when you run the Oracle Universal Installer.

When third-party vendor clusterware is present, Oracle Clusterware may interact with the third-party vendor clusterware. In using third-party vendor clusterware, note that Oracle Clusterware can integrate with third-party vendor clusterware for all operating systems except Linux and Windows. You need to have Oracle Cluster Registry and Voting Disks shared across all nodes.

### Installed Real Application Clusters Components

All instances in Real Application Clusters environment share the control file, server parameter file, redo log files, and all data files. These files reside on a shared cluster file system or on shared disks. Either of these types of file configurations are accessed by all the cluster database instances. Each instance also has its own set of redo log files. During failures, shared access to redo log files enables surviving instances to perform recovery.

## General System Installation Requirements for Real Application Clusters

Each node that is going to be part of your Real Application Clusters installation must meet the following hardware and software requirements. You will perform step-by-step tasks for hardware and software verification for the platform-specific pre-installation procedures.

### Hardware Requirements for Real Application Clusters Setup

Each node in a cluster requires the following hardware:

- External shared disks for storing the Oracle Clusterware files.

  Refer to the respective Real Application Clusters installation guide for information on the disk configuration options that are available. Review these options before you decide which storage option to use in your Real Application Clusters environment.

- One private Internet protocol (IP) address for each node to serve as the private interconnect. The following must be true for each private IP address:

  – It must be separate from the public network– It must be accessible on the same network interface on each node– It must have a unique address on each nodeThe private interconnect is used for inter-node communication by both Oracle Clusterware and Real Application Clusters. If the private address is available from a network name server (DNS), then you can use that name. Otherwise, the private IP address must be available in each node's `/etc/hosts` file on UNIX and `%SystemRoot%\system32\drivers\etc\hosts` on Windows.

  During Oracle Clusterware installation, the information you enter as the private IP address determines which private interconnects are used by Real Application Clusters database instances.

- One public IP address for each node, to be used as the Virtual IP address for client connections and for connection failover.

  This public Virtual IP address (VIP) must be associated with the same interface name on every node that is part of your cluster. In addition, the IP addresses that you use for all of the nodes that are part of a cluster must be from the same subnet. If you have a domain name server (DNS), then register the host names for the VIP with DNS. The Virtual IP address should not be in use at the time of the installation, because this is a Virtual IP address that Oracle manages.

- One public fixed hostname address for each node, typically assigned by the system administrator during operating system installation. If you have a DNS, then register both the fixed IP and the VIP address with DNS. If you do not have DNS, then you must make sure that both public IP addresses are in the node hostfile.

### Software Requirements for Real Application Clusters Setup

Each node in a cluster requires a supported interconnect software protocol to support Cache Fusion, and to support Oracle Clusterware polling. Your interconnect must be certified by Oracle for your platform. You should also have a Web browser, both to enable Oracle Enterprise Manager, and to view online documentation. For Oracle Database 10g requirements, Oracle Clusterware provides the same functionality as third-party vendor clusterware. Using Oracle Clusterware also reduces installation and support complications. However, you may require third-party vendor clusterware if you use a non-ethernet interconnect, or if you have deployed clusterware-dependent applications on the same cluster where you deploy Real Application Clusters.

## Cluster Setup and Pre-Installation Configuration Tasks for Real Application Clusters

Before installing Real Application Clusters, perform the following procedures:

1. Ensure that you have a certified combination of operating system and Oracle software version by referring to the OracleMetaLink certification information, which is located at the following Web site:

       http://metalink.oracle.com

   Click Certify & Availability, and select 1.View Certifications by Product.

   Note: The layout of the OracleMetaLink site and the site's certification policies are subject to change.

2. Configure a high-speed interconnect that uses a private network. Some platforms support automatic failover to an additional interconnect.

3. Determine the storage option for your system and configure the shared disk.

4. Ensure that the directory structure is similar in all nodes.

5. Ensure that all nodes in the cluster have the same time zone settings.

6. Install the operating system patches for the Oracle software. For more information refer to the respective Oracle software Real Application Clusters installation guide.

## Pre-Installation Tasks for Real Application Clusters on UNIX

You need to complete the following pre-installation tasks before you start the Oracle Universal Installer:

### Logging In to the System as root

Before you install the Oracle software, you must complete several tasks as the root user. To log in as the root user, complete one of the following procedures:

- If you are installing the software from an X Window System workstation or X terminal, then:

  1. Start a local terminal session, for example, an X terminal (xterm).

  2. If you are not installing the software on the local system, then enter the following command to enable remote hosts to display X applications on the local X server:

         $ xhost +

  3. If you are not installing the software on the local system, then use the ssh, rlogin, or telnet command to connect to the system where you want to install the software:

         $ telnet remote_host

  4. If you are not logged in as the root user, then enter the following command to switch user to root:

         $ su - root
         password:
         #

- If you are installing the software from a PC or other system with X server software installed, then:

  1. Start the X server software.

  2. Configure the security settings of the X server software to permit remote hosts to display X applications on the local system.

  3. Connect to the remote system where you want to install the software and start a terminal session on that system, for example, an X terminal (xterm).

  4. If you are not logged in as the root user on the remote system, then enter the following command to switch user to root:

         $ su - root
         password:
         #

### Creating Required UNIX Groups and Users

Depending on whether it is the first time an Oracle software is being installed on a system and on the products that you are installing, you may need to create several UNIX groups and a UNIX user account. The following UNIX group and users are required for all installations:

- The Oracle Inventory group (oinstall)

  You must create this group the first time you install Oracle software on the system. The usual name chosen for this group is oinstall. This group owns the Oracle inventory, which is a catalog of all Oracle software installed on the system.

  Note: If Oracle software is already installed on the system, the existing Oracle Inventory group must be the primary group of the UNIX user that you use to install new Oracle software. The following sections describe how to identify an existing Oracle Inventory group.

- The Oracle software owner user (oracle)

  You must create this user the first time you install Oracle software on the system. This user owns all of the software installed during the installation. The usual name chosen for this user is oracle. This user must have the Oracle Inventory group as its primary group.

You need to create other groups and users in addition to the ones created above. For more information on creating the required groups and users refer to the respective Oracle software Real Application Clusters installation guide.

#### Creating Identical Users and Groups on Other Cluster Nodes

The Oracle software users and groups must exist and be identical on all cluster nodes. To create these identical users and groups, you must identify the user ID and group IDs assigned to them on the node where you created them, then create the user and groups with the same name and ID on the other cluster nodes. You must create identical users and groups on other cluster nodes only if you are using local users and groups. If you are using users and groups defined in a directory service such as NIS, then they are already identical on each cluster node.

##### Identifying the User and Group IDs

To determine the user ID (UID) and the group IDs (GID) for the groups, follow these steps:

1. Enter following command:

       # id oracle

   The output from this command is similar to the following:

       uid=440(oracle) gid=200(oinstall) groups=201(dba),202(oper)

2. From the output, identify the user identity (UID) for the oracle user and the group identities (GIDs) for the groups to which it belongs.

##### Creating the User and Groups on the Other Cluster Nodes

To create the user and groups on the other cluster nodes, repeat the following procedure on each node:

1. Log in to the cluster node in which you want to create the user and groups as root.

2. Enter commands as per the syntax to create the respective groups. Use the -g option to specify the correct GID for each group:

       # /usr/sbin/groupadd -g <group_id> <group_name>

#### Configuring SSH on all Cluster Nodes

Before you install and use Oracle Real Application Clusters, you must configure secure shell (SSH) for the `oracle` user on all cluster nodes. Oracle Universal Installer uses the ssh and scp commands during installation to run remote commands on and copy files to the other cluster nodes. You must configure SSH so that these commands do not prompt for a password.

Note: This section describes how to configure OpenSSH version 3. If SSH is not available, then Oracle Universal Installer attempts to use rsh and rcp instead. However, these services are disabled by default on most Linux systems.

##### Configuring SSH on Cluster Member Nodes

To configure SSH, complete the following steps on each cluster node:

1. Log in as the oracle user.

2. If necessary, create the .ssh directory in the oracle user's home directory and set the correct permissions on it:

       $ mkdir ~/.ssh
       $ chmod 700 ~/.ssh

3. Enter the following commands to generate an RSA key for version 2 of the SSH protocol:

       $ /usr/bin/ssh-keygen -t rsa

   At the prompts:

   - Accept the default location for the key file.

   - Enter and confirm a pass phrase that is different from the oracle user's password.

   This command writes the public key to the `~/.ssh/id_rsa.pub` file and the private key to the `~/.ssh/id_rsa` file. Never distribute the private key to anyone.

4. Enter the following command to generate a DSA key for version 2 of the SSH protocol:

       $ /usr/bin/ssh-keygen -t dsa

   At the prompts:

   - Accept the default location for the key file.

   - Enter and confirm a pass phrase that is different from the oracle user's password.

   This command writes the public key to the `~/.ssh/id_dsa.pub` file and the private key to the `~/.ssh/id_dsa` file. Never distribute the private key to anyone.

5. Copy the contents of the `~/.ssh/id_rsa.pub` and `~/.ssh/id_dsa.pub` files to the `~/.ssh/authorized_keys` file on this node and to the same file on all other cluster nodes.

   Note: The `~/.ssh/authorized_keys` file on every node must contain the contents from all of the `~/.ssh/id_rsa.pub` and `~/.ssh/id_dsa.pub` files that you generated on all cluster nodes.

6. Change the permissions on the `~/.ssh/authorized_keys` file on all cluster nodes:

       $ chmod 600 ~/.ssh/authorized_keys

At this point, if you use ssh to log in to or run a command on another node, you are prompted for the pass phrase that you specified when you created the DSA key.

##### Enabling SSH User Equivalency on Cluster Member Nodes

To enable Oracle Universal Installer to use the ssh and scp commands without being prompted for a pass phrase, follow these steps:

1. On the system where you want to run Oracle Universal Installer, log in as the oracle user.

2. Enter the following commands:

       $ exec /usr/bin/ssh-agent $SHELL
       $ /usr/bin/ssh-add

3. At the prompts, enter the pass phrase for each key that you generated.

   If you have configured SSH correctly, then you can use the ssh or scp commands without being prompted for a password or a pass phrase.

4. To test the SSH configuration, enter the following commands from the same terminal session, testing the configuration of each cluster node:

       $ ssh nodename1 date
       $ ssh nodename2 date
       .
       .

   These commands should display the date set on each node. If any node prompts for a password or pass phrase, then verify that the `~/.ssh/authorized_keys` file on that node contains the correct public keys.

   Note: The first time you use SSH to connect to a node from a particular system, you might see a message stating that the authenticity of the host could not be established. Enter yes at the prompt to continue. You should not see this message again when you connect from this system to that node. If you see any other messages or text, apart from the date, then the installation can fail. Make any changes required to ensure that only the date is displayed when you enter these commands. You should ensure that any parts of login scripts that generate any output, or ask any questions, are modified so that they only act when the shell is an interactive shell.

5. To ensure that X11 forwarding will not cause the installation to fail, create a user-level SSH client configuration file for the Oracle software owner user, as follows:

   1. Using any text editor, edit or create the `~oracle/.ssh/config` file.

   2. Make sure that the ForwardX11 attribute is set to no, for example:

          Host *
          ForwardX11 no

6. You must run Oracle Universal Installer from this session or remember to repeat steps 2 and 3 before you start Oracle Universal Installer from a different terminal session.

##### Preventing Oracle Clusterware Installation Errors Caused by stty Commands

During an Oracle Clusterware installation, the Oracle Universal Installer uses SSH (if available) to run commands and copy files to the other nodes. During the installation, hidden files on the system (for example, `.bashrc` or `.cshrc`) can cause installation errors if they contain stty commands.

To avoid this problem, Oracle recommends that you modify these files to suppress all output on `STDERR`, as in the following examples:

- Bourne, Bash, or Korn shell:

      if [ -t 0 ]; then
      stty intr ^C
      fi

- C shell:

      test -t 0
      if ($status == 0) then
      stty intr ^C
      endif

  Note: When SSH is not available, the Installer uses the rsh and rcp commands instead of ssh and scp. If there are hidden files that contain stty commands that are loaded by the remote shell, then this error can also occur.

### Configuring the Oracle User Environment

You run Oracle Universal Installer from the oracle account. However, before you start Oracle Universal Installer you must configure the environment of the oracle user. For more information on configuring the environment, refer to the respective Oracle product Real Application Clusters installation guide.

### Checking the Hardware Requirements

Each system must meet certain minimum hardware requirements. For more information on hardware requirements, refer to the respective Oracle product Real Application Clusters installation guide.

### Checking the Network Requirements

There are some minimum networking hardware and Internet protocol (IP) address requirements. Check that you have the networking hardware and Internet protocol (IP) address required for an Oracle Real Application Clusters installation.

#### Network Hardware Requirements

Each node in the cluster must meet the following requirements:

- Each node must have atleast two network adapters; one for the public network interface and one for the private network interface (the interconnect).

- The interface names associated with the network adapters for each network must be the same on all nodes.

- For increased reliability, you can configure redundant public and private network adapters for each node.

- For the public network, each network adapter must support TCP/IP.

- For the private network, the interconnect must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP (Gigabit Ethernet or better recommended).

  Note: UDP is the default interconnect protocol for Real Application Clusters and TCP is the interconnect protocol for Oracle Clusterware. Token-Ring is not supported for the interconnect.

- For the private network, the end points of all designated interconnect interfaces must be completely reachable on the network.

#### Network Parameter Requirements

If you are using NFS, then you must set the values for the NFS buffer size parameters rsize and wsize to atleast 16384. Oracle recommends that you use the value 32768.

#### IP Address Requirements

Before starting the installation, you must identify or obtain the following IP addresses for each node:

- An IP address and an associated host name registered in the domain name service (DNS) for each public network interface.

- One unused virtual IP address and an associated virtual host name registered in DNS, or resolved in the hostfile, or both, that you will configure for the primary public network interface.

  The virtual IP address must be in the same subnet as the associated public interface. After installation, you can configure clients to use the virtual host name or IP address. If a node fails, then its virtual IP address fails over to another node.

- A private IP address and optional host name for each private interface.

  Oracle recommends that you use non-routable IP addresses for the private interfaces; for example: 10.*.*.* or 192.168.*.*. You can use the `/etc/hosts` file on each node to associate private host names with private IP addresses.

#### Node Time Requirements

Before starting the installation, ensure that each member node of the cluster is set as closely as possible to the same date and time. Oracle strongly recommends using the Network Time Protocol feature of most operating systems for this purpose, with all nodes using the same reference Network Time Protocol server.

### Checking Software Requirements

The Oracle Universal Installer performs checks on your system to verify that it meets the requirements listed for your platform. To ensure that these checks pass, verify the requirements before you start the Installer.

Depending on the Oracle products that you intend to install, verify that the required software is installed on the system. For more information on software requirements, refer to the respective Oracle product Real Application Clusters installation guide.

### Configuring Kernel Parameters

Each cluster node must meet the recommended kernel values. For information on verifying and setting the kernel parameters, refer to the respective Oracle product Real Application Clusters installation guide.

### Identifying Required Software Directories

You must identify or create the following directories for the Oracle software, as follows:

- Oracle Base Directory

- Oracle Inventory Directory

- Oracle Clusterware Home Directory

- Oracle Home Directory

The following subsections describe the requirements for these directories.

#### Oracle Base Directory

The Oracle base directory acts as a top-level directory for Oracle software installations. It is analogous to the `C:\Oracle` directory used for Oracle software installations on Windows systems. On UNIX systems, the Optimal Flexible Architecture (OFA) guidelines recommend that you use a path similar to the following for the Oracle base directory:

    /mount_point/app/oracle_sw_owner

In this example:

- mount_point is the mount point directory for the file system that will contain the Oracle software.

  The examples in this guide use `/u01` for the mount point directory. However, you could choose another mount point directory, `/oracle or /opt/oracle` for example.

- oracle_sw_owner is the operating system user name of the Oracle software owner, for example oracle.

You can use the same Oracle base directory for more than one installation or you can create separate Oracle base directories for different installations. If different operating system users install Oracle software on the same system, then each user must create a separate Oracle base directory. The following example Oracle base directories could all exist on the same system:

    /u01/app/oracle

    /u01/app/orauser

    /opt/oracle/app/oracle

Regardless of whether you create an Oracle base directory or decide to use an existing one, you must set the `ORACLE_BASE` environment variable to specify the full path to this directory.

Note: The Oracle base directory can be on a local file system or on an NFS file system on a certified NAS device. Do not create the Oracle base directory on an OCFS version 1 file system.

#### Oracle Inventory Directory

The Oracle Inventory directory (oraInventory) stores an inventory of all software installed on the system. It is required by, and shared by, all Oracle software installations on a single system. The first time you install Oracle software on a system, Oracle Universal Installer prompts you to specify the path to this directory. If you are installing the software on a local file system, then Oracle recommends that you choose the following path:

    oracle_base/oraInventory

If the Oracle base directory is on a cluster file system, or on an NFS file system on a NAS device, then you must specify a path for the Oracle Inventory directory on a local file system. The Oracle base directory must be on a local file system to enable all of the nodes to have separate inventories.

Oracle Universal Installer creates the directory that you specify, and sets the correct owner, group, and permissions for it. You do not need to create it.

Note: All Oracle software installations rely on the Oracle base directory. Make sure that you back it up regularly. Do not delete the Oracle base directory unless you have completely removed all Oracle software from the system.

#### Oracle Clusterware Home Directory

The Oracle Clusterware home directory is the directory where you choose to install the software for Oracle Clusterware. You must install Oracle Clusterware in a separate home directory. When you run Oracle Universal Installer, it prompts you to specify the path to this directory, as well as a name that identifies it. Oracle recommends that you specify a path similar to the following for the Oracle Clusterware home directory:

    /u01/crs/oracle/product/10.2.0/app

Note: Because you must change the permissions of all of the parent directories of the Oracle Clusterware home directory after installing the software to grant write access only to the root user, the Oracle Clusterware home directory must not be a subdirectory of the Oracle base directory.

#### Oracle Home Directory

The Oracle home directory is the directory where you choose to install the software for a particular Oracle product. You must install different Oracle products, or different releases of the same Oracle product, in separate Oracle home directories. When you run Oracle Universal Installer, it prompts you to specify the path to this directory, as well as a name that identifies it. The directory that you specify must be a subdirectory of the Oracle base directory. Oracle recommends that you specify a path similar to the following for the Oracle home directory:

    oracle_base/product/10.2.0/db_1

Oracle Universal Installer creates the directory path that you specify under the Oracle base directory. It also sets the correct owner, group, and permissions on it. You do not need to create this directory.

### Identifying or Creating an Oracle Base Directory

Before starting the installation, you must either identify an existing Oracle base directory or if required create one. For more information on identifying or creating an Oracle Base Directory, refer to the respective Oracle product Real Application Clusters installation guide.

### Creating the Clusterware Home Directory

Oracle Universal Installer creates the Oracle Clusterware home directory for you. Ensure before you start the installation that you provide sufficient disk space on a file system for the Oracle Clusterware directory, and the parent directory of the Oracle Clusterware directory space is writable by the Oracle user.

To create the Oracle Clusterware home directory and specify the correct owner, group, and permissions for it, follow these steps:

1. Enter commands similar to the following to create the recommended subdirectories in the mount point directory that you identified and set the appropriate owner, group, and permissions on them:

       # mkdir -p /mount_point/crs/oracle_sw_owner/product/10.2.0/crs
       # chown -R root:oinstall /mount_point/crs
       # chmod -R 775 /mount_point/crs/oracle_sw_owner

   If the mount point you identified is `/u01`, then the recommended Oracle Clusterware home directory path is as follows:

       /u01/crs/oracle/product/10.2.0/crs

2. If necessary, repeat the commands listed in the previous step to create the same directory on the other nodes in the cluster.

3. Enter commands similar to the following to set the `ORACLE_BASE` and `ORACLE_HOME` environment variables in preparation for the Oracle Clusterware installation:

   - Bourne, Bash, or Korn shell:

         $ ORACLE_BASE=/u01/app/oracle
         $ ORACLE_HOME=/u01/crs/oracle/product/10.2.0/crs
         $ export ORACLE_BASE
         $ export ORACLE_HOME

   - C shell:

         % setenv ORACLE_BASE /u01/app/oracle
         % setenv ORACLE_HOME /u01/crs/oracle/product/10.2.0/crs

4. Enter the following commands to ensure that the `TNS_ADMIN` environment variable is not set:

   - Bourne, Bash, or Korn shell:

         $ unset TNS_ADMIN

   - C shell:

         % unsetenv TNS_ADMIN

5. To verify that the environment has been set correctly, enter the following commands:

       $ umask
       $ env | more

   Verify that the umask command displays a value of 22, 022, or 0022 and the environment variables that you set in this section have the correct values.

## Pre-Installation Tasks for Real Application Clusters on Windows

You need to complete the following pre-installation tasks before you start the Oracle Universal Installer to install Oracle Clusterware and Oracle Real Application Clusters on Microsoft Windows systems.

### Checking Hardware and Software Certification

Oracle recommends you to review the certification matrix on the OracleMetaLink Web site for the most up-to-date list of certified hardware platforms and operating system versions for the Oracle software. This Web site also provides patches and workaround information for bugs. For more information on accessing the OracleMetaLink site, refer to "Download and Install Patches".

### Checking the Hardware Requirements

Each system must meet minimum hardware requirements. For more information on hardware requirements and the steps involved to check them, refer to the respective Oracle product Real Application Clusters installation guide.

#### Hard Disk Space Requirements

Each system must meet certain hard disk space requirements. For more information on hard disk space requirements, refer to the respective Oracle product Real Application Clusters installation guide.

### Checking the Software Requirements

Each system must meet minimum software requirements. For more information on software requirements and the steps involved to check them, refer to the respective Oracle product Real Application Clusters installation guide.

### Checking the Network Requirements

There are some minimum networking hardware and Internet protocol (IP) address requirements. Check that you have the networking hardware and Internet protocol (IP) address required for an Oracle Real Application Clusters installation. For more information on the steps involved in checking these requirements, refer to the respective Oracle product Real Application Clusters installation guide.

#### Network Hardware Requirements

Each node in the cluster must meet the following requirements:

- Each node must have atleast two network adapters; one for the public network interface and one for the private network interface (the interconnect).

- The private and public network interface names must be different from each other and cannot contain any multibyte language characters. The names are case-sensitive.

- The private network interface name must be the same on all nodes.

- The public network interface name must be the same on all nodes.

- The public interface must be listed first in the ipconfig list.

- Oracle supports the TCP/IP protocol for the public and private networks.

#### IP Address Requirements

Before starting the installation, you must identify or obtain the following IP addresses for each node:

- An IP address and an associated host name registered in the domain name service (DNS) for each public network interface.

- One unused virtual IP address and an associated virtual host name registered in DNS, that you will configure for the primary public network interface.

  The virtual IP address must be in the same subnet as the associated public interface. After installation, you can configure clients to use the virtual host name or IP address. If a node fails, then its virtual IP address fails over to another node.

- A private IP address and optional host name for each private interface.

  Oracle recommends that you use private network IP addresses for the private interfaces; for example: 10.*.*.* or 192.168.*.*. You can use the `%SystemRoot%\system32\drivers\etc\hosts` file on each node to associate private host names with private IP addresses.

### Checking Individual Component Requirements

Depending on the Oracle products that you intend to install, verify the required components for the Oracle software you are installing and ensure that it is installed on the system. You will find more information on the components in the respective Oracle product Real Application Clusters installation guide.

## Configuring Storage for Oracle Clusterware

You need to perform certain storage configuration tasks before you start Oracle Universal Installer. For detailed information on the storage configuration tasks, refer to the respective Oracle product Real Application Clusters installation guide.

## Installing Oracle Clusterware on UNIX

This section provides you with information about how to use the Oracle Universal Installer to install Oracle Clusterware on UNIX.

### Running the Oracle Universal Installer to Install Oracle Clusterware

Complete the following steps to install Oracle Clusterware on your cluster. At any time during installation, if you have a question about what you are being asked to do, click Help in the Oracle Universal Installer page.

1. Start the `runInstaller` command with the `-formCluster` option from the main directory on the Oracle Clusterware 10g Release 2 (10.2) CD-ROM. This is a separate CD that contains the Oracle Clusterware software. When the Oracle Universal Installer displays the Welcome page, click Next.

2. Oracle Universal Installer displays the Installation Prerequisite Checks page. This screen shows the name, type, and status for all pre-requisite checks designed for the installation. Click Next.

3. Oracle Universal Installer displays the Specify Cluster Nodes for Node Addition page. Specify public, private, virtual IPs in this page and click Next.

4. Oracle Universal Installer displays the Specify Local Host Name page. Specify atleast one public and one private interface in this page and click Next.

5. Specify Oracle Cluster Registry location option to give a redundant OCR location and click Next.

6. Specify VD (Voting Disk Location) location option to give redundant VD locations and click Next.

7. Oracle Universal Installer displays the Summary page. Click Install and proceed.

8. After the installation is done, Oracle Universal Installer prompts you to run `orainstRoot.sh` and `root.sh`. You need to run these scripts and click OK.

9. After you run `root.sh` on all of the nodes, the Oracle Universal Installer runs the Oracle Notification Server Configuration Assistant and Oracle Private Interconnect Configuration Assistant. These assistants run without user intervention.

10. Oracle Universal Installer displays the End of Installation page. Click Exit.

### Running the Oracle Universal Installer in Silent Mode to Install Oracle Clusterware

You can run the Oracle Universal Installer in silent mode to install Oracle Clusterware by entering the following command:

    ./runInstaller -silent -responseFile <response file path>

For Oracle Clusterware, you need to use the -formCluster flag. You can pass session variables like nodelist, ocr, voting disk location etc from the command line for silent installation.

### Oracle Clusterware Background Processes

The following processes must be running in your environment after the Oracle Clusterware installation for Oracle Clusterware to function:

- oprocd — Process monitor for the cluster.

- evmd — Event manager daemon that starts the racgevt process to manage call-outs.

- ocssd — Manages cluster node membership and runs as oracle user; failure of this process results in node restart.

- crsd — Performs high availability recovery and management operations such as maintaining the OCR. Also manages application resources and runs as root user and restarts automatically upon failure.

## Installing Oracle Clusterware on Windows

This section provides you with information about how to use the Oracle Universal Installer to install Oracle Clusterware on Windows.

### Running the Oracle Universal Installer to Install Oracle Clusterware

Complete the following steps to install Oracle Clusterware on your cluster. At any time during installation, if you have a question about what you are being asked to do, click Help in the Oracle Universal Installer page.

1. Log in to Windows with Administrative privileges and run the `setup.exe` command on the Oracle Clusterware media. This will open the Oracle Universal Installer Welcome page.

2. After you click Next in the Welcome page, the Specify File Locations page will allow you to accept the displayed path name for the Oracle Clusterware products or select a different one. You may also accept default directory and path name for the location of your Clusterware home or browse for an alternate directory and destination. You must select a destination that exists on each cluster node that is part of this installation. Click Next to confirm your choices and proceed to the Language Selection page.

3. Select the language or languages for your Oracle Clusterware installation in the Language Selection page, then click Next for the Cluster Configuration page.

4. The Cluster Configuration page contains predefined node information if the Oracle Universal Installer detects that your system has the Oracle9i Release 2 Clusterware. Otherwise, the Oracle Universal Installer displays the Cluster Configuration page without predefined node information.

   Provide your own cluster name if you do not wish to use the name provided by the Oracle Universal Installer. Note that the selected cluster name must be globally unique throughout the enterprise and its allowable character set is the same as that for hostnames, that is, underscores (_), hyphens (-), and single-byte alphanumeric characters (a to z, A to Z, and 0 to 9).

   Enter a public and a private host name for each node. Neither the public nor the private name should have a domain qualifier. When you enter the public host name, use the primary host name of each node, that is, the name displayed by the hostname command. The private node refers to an address that is only accessible by the other nodes in this cluster, and which Oracle uses for Cache Fusion processing. You may enter either a private host name or a private IP address for each node.

   Click Next after you have entered the cluster configuration information. This saves your entries and opens the Specify Network Interface Usage page.

5. In the Specify Network Interface Usage page the Oracle Universal Installer displays a list of cluster-wide interfaces. Use the drop-down menus on this page to classify each interface as Public, Private, or Do Not Use. The default setting for each interface is Do Not Use. You must classify atleast one interconnect as Public and one as Private. Click Next when you have made your selections to open the Select Disk Formatting Options page.

6. In the Select Disk Formatting Options page, you indicate what OCFS you want the Oracle Universal Installer to format for you.

   The formatting options are as follows:

   - Format two logical drives for data and software storage. In this case, the Oracle Universal Installer creates two cluster file systems, one for the database files and one for the Oracle home. The OCR and voting disk are created in the datafile directory.

   - Format one logical drive for software storage. In this case, the Oracle Universal Installer creates one cluster file system for the Oracle home and requires two additional partitions, one for the OCR and one for the voting disk. If you select this option, then you must have configured the partitions before proceeding with the installation. These partitions will be stamped with "ocrcfg" for the OCR and "votedsk" for the voting disk.

   - Format one logical drive for data file storage. In this case, the Oracle Universal Installer creates one cluster file system for the database files which is also used to store the OCR and voting disk.

   - Do not format any logical drives. In this case, the Oracle Universal Installer requires two partitions, one for the OCR and one for the voting disk. If you select this option, then you must have configured the partitions before proceeding with the installation. These partitions will be stamped with "ocrcfg" for the OCR and "votedsk" for the voting disk.

   Click Next after making your selection in the Select Disk Formatting Options page. The next page that the Oracle Universal Installer displays depends on your selection on the Select Disk Formatting Options page, as described in the following step.

7. If you selected the "Format two logical drives for data and software storage" option on the Select Disk Formatting Options page, then you must complete the Select Software Storage Drive page and the Select Data Storage Drive page as described in Steps a and b respectively.

   If you selected the "Format one logical drive for software storage" option on the Select Disk Formatting Options page, then you must complete the Select Software Storage Drive page, the Disk Configuration - Oracle Cluster Registry (OCR) page, and the Disk Configuration - Voting Disk page as described in Steps a, c, and d respectively.

   If you selected the "Format one logical drive for data file storage" option on the Select Disk Formatting Options page, then you must complete the Select Data Storage Drive page as described in Step b.

   If you selected the "Do not format any logical drives" option on the Select Disk Formatting Options page, then you must complete the Disk Configuration - Oracle Cluster Registry (OCR) page and the Disk Configuration - Voting Disk page as described in Steps c and d respectively.

   1. On the Select Software Storage Drive page, choose a shared drive to see a list of available partitions on that drive. Choose a partition with sufficient space to hold your Oracle home and select the partition's disk number and partition number from the list. Click Next to proceed.

   2. On the Select Data Storage Drive page, choose a shared drive to see a list of available partitions on that drive. Choose a partition with sufficient space to hold your database files and select the partition's disk number and partition number from the list. Click Next to proceed.

   3. On the Disk Configuration - Oracle Cluster Registry (OCR) page, choose a partition with sufficient space to hold your OCR and select the partition's disk number and partition number from the list. Click Next to proceed.

   4. On the Disk Configuration - Voting Disk page, choose a partition with sufficient space to hold your voting disk and select the partition's disk number and partition number from the list. Click Next to proceed.

      Note: The Oracle Universal Installer pages described in this step display logical drives from which you must make your selections. To be valid for selection, a logical drive must be located on a disk without a primary partition.

8. After you click Next, the Oracle Universal Installer checks whether the remote inventories are set. If they are not set, then the Oracle Universal Installer sets up the remote inventories by setting registry keys. The Oracle Universal Installer also verifies the permissions to enable writing to the inventory directories on the remote nodes. After completing these actions, the Oracle Universal Installer displays a Summary page that shows the cluster node information along with the space requirements and availability. Verify the installation that the Oracle Universal Installer is about to perform and click Finish.

9. When you click Finish, the Oracle Universal Installer installs the OCFS and Oracle Clusterware software on the local node and validates the installation again. The Oracle Universal Installer will also create any required OCFS file systems. After validating the installation, the Oracle Universal Installer completes the Oracle Clusterware software installation and configuration on the remote nodes.

### Running the Oracle Universal Installer in Silent Mode to Install Oracle Clusterware

You can run the Oracle Universal Installer in silent mode to install Oracle Clusterware by entering the following command:

    ./setup.exe -silent -responseFile <response file path>

For Oracle Clusterware, you need to use the `-formCluster` flag. You can pass session variables like nodelist, ocr, voting disk location etc from the command line for silent installation.

### Oracle Clusterware Background Processes

The following processes must be running in your environment after the Oracle Clusterware installation for Oracle Clusterware to function:

- oprocd — Process monitor for the cluster.

- evmd — Event manager daemon that starts the racgevt process to manage call-outs.

- ocssd — Manages cluster node membership and runs as oracle user; failure of this process results in node restart.

- crsd — Performs high availability recovery and management operations such as maintaining the OCR. Also manages application resources and runs as root user and restarts automatically upon failure.

## Adding Additional Nodes to the Cluster for Mass Deployment

If you want to add additional nodes to the cluster by copying the Oracle Clusterware home to another node in the cluster, then complete the following procedure:

1. Perform a single-node cluster Oracle Clusterware installation on node1. In the following steps, the variable `$CRS_HOME` represents the Oracle Clusterware home directory of the successfully installed Oracle Clusterware software.

2. Go to the directory `$CRS_HOME/oui/bin`, and run the script `addNode.sh`.

3. The node selection page comes up. Select the nodes to be added and click Next.

4. The Summary page opens. Verify if the Summary page contains the correct node information, and click Next.

5. When prompted, run the script `rootaddnode.sh` on your local node.

6. If prompted to do so, run the script `orainstroot.sh` on the node on which you are creating another Oracle Clusterware home directory.

7. Run the `root.sh` script on the node on which you are creating another Oracle Clusterware home directory.

8. From the Oracle Clusterware home directory you created on the additional node, in the path `$CRS_HOME/bin`, run the following command:

       ./racgons add_config node2:4948

   In the preceding syntax example, the variable node2 is the name of the node on which you are configuring the additional Oracle Clusterware home directory.

## Installing Product Software on a Cluster

Once clusterware has been installed and pre-installation tasks have been completed, you are ready to install the cluster-ready product on the cluster. To do this, you may use Oracle Universal Installer in interactive mode, command line mode, or silent mode using a response file.

See Also: Refer to "Modes of Installation" on page 2-3 for more information on the different modes.

### Cluster Detection

When Oracle Universal Installer detects that the destination you entered on the File Locations page is part of a cluster, one of two dialogs will appear:

- If the Oracle home name entered is new, the Specify Hardware Installation Mode page appears. Use this page to select between cluster and non-cluster installation:

  - Cluster installation: Select the nodes for your installation from the list of nodes on which clusterware has been installed.

  - Non-cluster installation: Install on a single-node only, even though that node is part of a cluster.

- If the Oracle home name entered is a pre-existing home, the Selected Nodes page appears. This is an information-only page that displays the nodes associated with the Oracle home. The installation will be performed on all nodes selected. If any of the selected nodes are down, then the installation will not be allowed to proceed.

  Note: In Oracle Universal Installer, click Installed Products to display the Inventory panel, which lists all installed Oracle homes. Clustered Oracle homes, homes installed on multiple cluster nodes, have an attribute called Cluster Nodes, which lists all the nodes associated with that home.

### Availability Checking

When the user clicks Next on the Specify Hardware Installation Mode page or Selected Nodes page, Oracle Universal Installer performs the following availability checks on the nodes:

- Network reachability: Tests whether the remote cluster nodes are up and running.

- Network configuration: Tests whether the remote cluster nodes are properly network configured.

- Inventory setup: Tests whether the inventory is set up on the remote nodes. If not set up, on UNIX, Oracle Universal Installer prompts the user to run root scripts on the remote nodes to set up inventory. On Windows, Oracle Universal Installer sets up the inventory location in the Windows registry.

- Inventory permission: Tests whether the Central Inventory location is writable by the user.

- Oracle home permission: Tests whether the Oracle home is writable by the user.

If all selected nodes are available, the installation process continues to the next step. If a node (or nodes) is not available, the page is re-displayed with a Status column indicating the results of the check. The user must fix the problem or choose another set of nodes in order to proceed.

If any of the remote nodes is unreachable, check if those nodes are up and running and if they are properly network configured. If either the inventory or the Oracle home is not writable, check for the appropriate permissions on the remote nodes for these directories.

### Cluster Installation

In a typical cluster installation, when clusterware is present, Universal Installer installs the Oracle software onto the node on which Oracle Universal Installer is running, then propagates the Oracle home from the local node installation to the other nodes that are part of the installation. Then Universal Installer runs `-attachHome` on the remote nodes to update the inventory. After this, you need to run `orainstRoot.sh` (if required) and `root.sh` on the local and remote nodes. After the scripts are run, the config steps are executed.

If the cluster is Cluster File System (CFS) or Network File System (NFS) mounted, then Oracle Universal Installer does not propagate the Oracle home to other nodes, as files are shared across nodes and the installation will exist on the shared disk.

Note: For cluster installations, you must run `oraInstRoot.sh` on each node of the cluster to set up the inventory.

### Cluster Installation in Silent Mode

For Real Application Clusters installation, you can specify `-local` flag, `CLUSTER_NODES, REMOTE_NODES,` and `LOCAL_NODE` session variables to indicate the nodes on which the installation needs to be done.

When you use the `-local` flag in a cluster installation, it means that the installation or inventory operation needs to be performed only on the local node. The `CLUSTER_NODES` session variable indicates that the specified nodes are the part of the cluster.

    ./runInstaller -local "CLUSTER_NODES={a,b,c,d}"

In the above example, the installation would be performed in the local node and nodes a, b, c, and d would be taken as the part of the cluster.

When you use the `REMOTE_NODES` session variable in a cluster installation, it means that the installation or the inventory operation needs to be performed on the local node and the other nodes apart from the local node.

    ./runInstaller "REMOTE_NODES={c,d}" "CLUSTER_NODES={a,b,c,d}"

In the above example, the installation would be performed on the local node a and nodes c and d. The nodes a, b, c, and d would be taken as the part of the cluster.

### Cluster De-installation

When removing a clustered Oracle home, Oracle Universal Installer first removes software from the node from which you are de-installing, then removes software from the other cluster nodes associated with the Oracle home.

## Command Line Options for Cluster Installs (UNIX Only)

When using Oracle Universal Installer in suppressed or silent mode, specify the path to the remote copy or shell program on the local node to use for cluster installs using the following optional flags:

    -remoteShell <path to program>
    -remoteCopy <path to program>

The default for `remoteShell` is `/usr/local/bin/ssh`. The default for `remoteCopy` is `/usr/local/bin/scp`. If these are not present, Oracle Universal Installer defaults to `rsh` and `rcp` respectively.

## Patchsets and Upgrades

Oracle Universal Installer enables you to upgrade a product from one version to another version. An upgrade is a major product enhancement that often requires installation of the upgraded software. For example, if you convert your Oracle9i Database to Oracle 10gR2 Database, then it is called an upgrade.

A group of patches form a patchset. For example, if you convert your Oracle 10gR1 from version 10.1.1 to version 10.1.2, then it is called applying a patchset.

## Post Installation Tasks

After completing a Real Application Clusters installation, it is advisable that you perform the following tasks:

### Verification

After your Oracle product with Real Application Clusters installation is complete, go to `$ORACLE_HOME/Opatch` on each node and execute the following command:

    $ORACLE_HOME/OPatch/opatch lsinventory -detail

This command lists the components installed inside the Oracle home and also lists the local and remote nodes for the home. Verify this information. You may also want to look at installation logs, `inventory.xml`, and `oraclehomeproperties.xml`.

Refer to "Troubleshooting Real Application Clusters / Oracle Clusterware Installation" in case the check fails.

### Back Up Disk after Installation

After your Oracle product with Real Application Clusters installation is complete and after you are sure that your system is functioning properly, make a backup of the contents of the disk.

Also make a backup of the disk contents after you complete any node additions or node deletions and after running any de-installation procedures.

### Download and Install Patches

Refer to the OracleMetaLink Web site for required patches for your installation. To download the required patches:

1. Use a Web browser to view the OracleMetaLink Web site:

       http://metalink.oracle.com

2. Log in to OracleMetaLink.

3. On the main OracleMetaLink page click Patches.

4. On the Select a Patch Search Area page click New MetaLink Patch Search.

5. On the Simple Search page click Advanced.

6. On the Advanced Search page click the search icon next to the Product or Product Family field.

7. In the Search and Select: Product Family field, enter the Oracle product name in the For field and click Go.

8. Select the Oracle product under the Results heading and click Select. The product name appears in the Product or Product Family field and the current release appears in the Release field.

9. Select your platform from the list in the Platform field and click Go.

10. Any available patches appear under the Results heading.

11. Click the number of the patch that you want to download.

12. On the Patch Set page, click View README and read the page that appears. The README page contains information about the patch set and how to apply the patches to your installation.

13. Return to the Patch Set page, click Download, and save the file on your system.

14. Use the unzip utility provided with Oracle software to uncompress the Oracle patches that you downloaded from OracleMetaLink. The unzip utility is located in the `$ORACLE_HOME/bin` directory.

    Note: By default the patches would only be applied on the nodes on which the Oracle software installation was done. The node list can be changed using `-enableRollingUpgrade` option.

### Configure Oracle Products

Many Oracle products and options must be configured before you use them for the first time. Before using individual Oracle products or options, refer to the manual in the product documentation library which is available on the documentation CD-ROM or on the OTN Web site.

## De-installing Real Application Clusters Software

If you need to de-install Real Application Clusters software, then you must run the Oracle Universal Installer to de-install the software on the same node from which you performed the installation. You must first de-install the Oracle product before de-installing the Oracle Clusterware software.

Perform the following procedures to de-install Oracle product and Oracle Clusterware software, as described in the following sections.

### De-installing Oracle Product Software on UNIX

The following steps describe how to use Oracle Universal Installer to remove Oracle software from an Oracle home on UNIX systems:

Note: Always use Oracle Universal Installer to remove Oracle software. Do not delete any Oracle home directories without first using Oracle Universal Installer to remove the software.

1. If necessary, log in as the `oracle` user:

       $ su - oracle

2. Set the `ORACLE_HOME` environment variable to specify the path of the Oracle home directory that you want to remove:

   - Bourne, Bash, or Korn shell:

         $ ORACLE_HOME=/u01/app/oracle/product/10.2.0/db_1
         $ export ORACLE_HOME

   - C shell:

         $ setenv ORACLE_HOME /u01/app/oracle/product/10.2.0/db_1

3. Stop all processes running in this Oracle home.

4. Start Oracle Universal Installer with the `-deinstall` flag. When you use this flag only, the de-installation happens on the specified components. The Oracle home does not get removed from the Central Inventory. This is equivalent to selecting only the components and clicking Deinstall in the GUI. The syntax is as follows:

       ./runInstaller -deinstall ORACLE_HOME=<LOCATION_OF_ORACLE_HOME>

   You can use `REMOVE_HOMES` variable to specify the location of the Oracle home to be removed. When you use this variable, the de-installation happens for the specified components and the Oracle home gets removed from the Central Inventory. This is equivalent to selecting an Oracle home and clicking Deinstall in the GUI. The syntax is as follows:

       ./runInstaller -deinstall ORACLE_HOME=<LOCATION_OF_ORACLE_HOME> "REMOVE_
       HOMES={<LOCATION_OF_ORACLE_HOME_TO_BE_REMOVED>}"

   You can also use `-removeallfiles` flag with the `REMOVE_HOMES` variable to completely de-install the components, remove the Oracle home from the Central Inventory and delete the Oracle home directory. The syntax is as follows:

       ./runInstaller -deinstall ORACLE_HOME=<LOCATION_OF_ORACLE_HOME> "REMOVE_
       HOMES={<LOCATION_OF_ORACLE_HOME_TO_BE_REMOVED>}" -removeallfiles

   For detailed instructions on de-installing an Oracle product, refer to the respective Oracle product installation guide.

### De-installing Oracle Product Software on Windows

The following steps describe how to use Oracle Universal Installer to remove Oracle software from an Oracle home on Windows systems:

Note: Always use Oracle Universal Installer to remove Oracle software. Do not delete any Oracle home directories without first using Oracle Universal Installer to remove the software.

1. Stop all the Oracle services running in this Oracle home.

2. Start Oracle Universal Installer: from the Start menu, select Programs, then ORACLE_ HOME_NAME, then Oracle Installation Products, then Oracle Universal Installer.

   The Welcome screen for Oracle Universal Installer appears.

3. Click Deinstall Products.

   The Inventory screen appears.

4. Expand the tree of installed components until you find the components to remove. For example, if you installed a database with the Enterprise Edition option and later installed additional components with the Custom option, expand the Oracle home component to display all the components installed in the Oracle home.

5. Select the components to remove.

6. Click Remove.

   The Confirmation screen appears.

7. Click Yes to remove the selected components to initiate the removal process.

8. After the components are removed from your computer, the Inventory screen appears without the removed components.

9. Click Close to close the Inventory screen.

10. Click Cancel to exit Oracle Universal Installer.

11. Click Yes to confirm that you want to exit.

    For detailed instructions on de-installing an Oracle product, refer to the respective Oracle product installation guide.

### De-installing Oracle Clusterware

Before de-installing Oracle Cluster software ensure that you have removed the respective Oracle product's home in each node by following the steps in the preceding section. The following section explains de-installation on Windows and UNIX.

#### De-installing Oracle Clusterware from UNIX Environment

De-install each Oracle product Real Application Clusters home by running the procedure in the previous section. Then complete the de-installation by removing the Oracle Clusterware software using the following procedures:

1. Run the command `$CRS_HOME/install/rootdelete.sh` to disable the Oracle Clusterware applications that are running on the cluster node. The `rootdelete.sh` script requires three arguments. If you are running this command on a remote node of the cluster, then use `remote` as the first argument, otherwise use `local` as the first argument. If the `ocr.loc` file is on a shared file system, then use `sharedvar`. Otherwise use `nosharedvar` as the second argument. If the Oracle Clusterware home is on a shared file system, then use `sharedhome`, otherwise use `nosharedhome` as the third argument. Repeat this step on each node of the cluster from which you want to de-install Oracle Clusterware.

2. Run the script `$CRS_HOME/install/rootdeinstall.sh` on a local node to remove the OCR. You only need to run this script once.

3. Start Oracle Universal Installer. In the Welcome page, click Deinstall Products to display the list of installed products. Select the Oracle Clusterware home to de-install.

#### De-installing Oracle Clusterware from Windows Environment

De-install each Oracle product Real Application Clusters home by running the procedure in the previous section. Then complete the de-installation by removing the Oracle Clusterware software using one of the following procedures:

- De-installing Oracle Clusterware with No Previous Cluster Software Versions.

- De-installing Oracle Clusterware with Clusterware Downgrade to 9.2.

##### De-installing Oracle Clusterware with No Previous Cluster Software Versions

Perform the following steps to de-install Oracle 10g Clusterware software from a Windows environment:

1. Click Start and navigate to Settings, then to Control Panel, then to Administrative Tools, and then to Services. Stop the service `oracleremexec`.

2. Start the Oracle Universal Installer. On the Welcome page, click Deinstall Products to display the list of installed products. Select the Oracle Clusterware home you want to de-install.

3. If you have services with names such as `OracleCRSTokenname`, then remove them by running the following command:

       crsuser remove user_name
       where user_name is a user name.

4. Shut down and restart each node that is a member of your cluster.

5. If you are not using a cluster file system, then on each node, use Windows Explorer to delete the Oracle directory, its subdirectories, and their contents.

##### De-installing Oracle Clusterware with Clusterware Downgrade to 9.2

Perform the following steps to de-install Oracle 10g Clusterware software from a Windows environment that also has 9.2 Real Application Clusters:

1. Run `$CRS_HOME\bin\GuiOracleOBJManager.exe` to make sure that the symbolic link named `srvcfg` exists and points to a disk partition, if you are not using OCFS to store the OCR.

2. If Oracle9i release 9.2 Clusterware uses Oracle CFS (Cluster File System), then ensure the following is true for all nodes:

   - The registry key `HKLM\SOFTWARE\Oracle\osd9i\ocr` existsThe registry key `HKLM\SOFTWARE\Oracle\osd9i\ocr` has a string value `CfsOcrRoot` that points to a release 9.2 OCR (CFS) location

3. Run `$CRS_HOME\bin\ocrconfig -downgrade` to downgrade the Cluster Registry to a release 9.2 OCR.

4. On each node, copy `$CRS_HOME\cfs\OcfsFindVol.exe` to `%SYSTEMROOT%\system32\osd9i\cfs`.

5. Run `$CRS_HOME\oui\bin\setup.exe` to start Oracle Universal Installer. On the Welcome page click Deinstall Products to list all the installed products. Select the Oracle Clusterware home name from the displayed products, and click Remove to deconfigure and de-install the product.

6. On each node, run `%SYSTEMROOT%\system32\osd9i\olm\OracleOBJService.exe /install` to re-install the Oracle 9.2 object service. Then start Oracle Object Service.

7. On each node, run `%SYSTEMROOT%\system32\osd9i\cfs\OcfsFindVol.exe /i:%SYSTEMROOT%\system32\osd9i\cfs\OcfsFindVol.exe` to re-install the Oracle Cluster Volume service. Then start the Oracle Cluster Volume service.

8. From a 9.2 Real Application Clusters Oracle home on each node run the command `ORACLE_HOME\bin\gsdservice.exe -install`. Then start the `OracleGSDService`.

9. On each node, copy `%SYSTEMROOT%\system32\osd9i\orafencedrv.sys %SYSTEMROOT%\system32\drivers\orafenceservice.sys`.

## Converting Single Instance Nodes to Real Application Clusters

You can convert single instance nodes to Real Application Clusters using Oracle Universal Installer. For details on converting single instance nodes to Real Application Clusters, refer to the Oracle Clusterware and Oracle product Real Application Clusters installation guide for your platform.

## Troubleshooting Real Application Clusters / Oracle Clusterware Installation

The following flowchart explains how you can solve problems associated with the Real Application Clusters or Oracle Clusterware installation.

Figure 6-1 Troubleshooting Real Application Clusters / Oracle Clusterware Installation